Method of repairing scratches in digital images

ABSTRACT

A method of repairing scratches in a digital image includes counting the number of abnormal pixels among neighboring pixels of a selected pixel. The selected pixels that are surrounded by abnormal pixels in a number greater than a predetermined value are designed as particular pixels. An area surrounding each particular pixel is divided into a plurality of blocks. Furthermore, brightness difference between two of the blocks are calculated. Scratch pixels are found from the particular pixels based on brightness difference between blocks. Then, an area surrounded by scratch pixels are subdivided and filled up. Thereby, the scratch pixels can be precisely found and repaired without the need of hardware.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to an image processing method, and moreparticularly to a method of repairing scratches in digital images.

2. Related Art

As digital technology progresses, the way to process images hassignificantly changed. Much more information can be processed, storedand transmitted digitally. The digital images are formed by means ofdigital cameras or image scanners, or directly generated by drawingsoftware applications. The quality of digital images is based on thequality of the image originals and the quality of the used input/outputequipment. Good performance of the input equipment greatly influencesthe quality of the acquired image.

The image original, which may be for example, a document sheet, may bescratched or damaged by dust or fine particles. Therefore, the acquireddigital image usually needs to be repaired to recover its intactappearance.

Currently, repairing images is mainly performed on films, for example,an ‘automatic scratch remover’, which is a new technology in thescanning art. A film scanner is a high level electronic image-relatedproduct that needs advanced photoelectric technology and high productionquality.

In the present state of the art, scratches are found by hardware,resulting in non-satisfactory repairing with high processing costs.Applications such as PHOTOSHOP repair the damaged image on the film witha complicated operation and are substantially time-consuming. Usually ittakes several hours to repair a fine scratch. Such an image repairprocess is not efficient.

Furthermore, the method of processing scratches in the digital imageincludes finding out where the scratches are located, and repairingthem. Presently, localization of the scratches is more difficult thanrepairing the scratches. Therefore, there is a need for a method ofrepairing scratches in digital images, which can precisely locate thescratches.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to provide a method ofrepairing scratches in a digital image, and precisely find thescratches. Further, to repair the images without the need of hardware.

In order to achieve the above and other objectives, the method ofrepairing scratches in a digital image includes calculating a brightnessdifference between a selected pixel and one of neighboring pixels thatsurround the selected pixel. The number of abnormal pixels among theneighboring pixels then is counted. The selected pixels that aresurrounded by the abnormal pixels in a number more than a predeterminedvalue are designated as particular pixels. An area surrounding eachparticular pixel then is divided into a plurality of blocks. Thebrightness difference between the blocks then is calculated. The scratchpixels are distinguished from the particular pixels based on brightnessdifference between blocks. Lastly, an area surrounded by scratch pixelsis subdivided and filled up.

According to the invention, the scratches are found rapidly andprecisely without the need of hardware. The results provided by themethod of the invention are better, while the processing costs arereduced.

Further scope of applicability of the present invention will becomeapparent from the detailed description given hereinafter. However, itshould be understood that the detailed description and specificexamples, while indicating preferred embodiments of the invention, aregiven by way of illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art from this detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description given herein below illustration only, and is thusnot limitative of the present invention, and wherein:

FIG. 1 is a schematic view illustrating a method of repairing scratcheson a digital image according to the invention;

FIG. 2 is a flowchart of a method of repairing scratches on a digitalimage according to one embodiment of the invention;

FIG. 3 is a schematic view of neighboring pixels of a selected pixelaccording to the invention; and

FIG. 4A˜FIG. 4D is a schematic view of blocks of an area surrounding aparticular pixel according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

The invention is illustrated with reference to the accompanied drawings.

A method of repairing scratches in a digital image according to theinvention is implemented to repair digital images captured by digitalcameras, scanners, or formed by drawing software applications.

FIG. 1 is a schematic view illustrating a method of repairing scratcheson a digital image according to the invention. First, calculatingbrightness difference between a selected pixel Y (i, j) and neighboringpixels (step 110). Then count the number of abnormal pixels among theseneighboring pixels (step 120). When the number of abnormal pixels isgreater than a predetermined value, the pixels in question aredesignated as particular pixels (step 130). Peripheral areas around theparticular pixels are divided into a plurality of blocks, and brightnessdifferences between adjacent blocks are estimated (step 140). Scratchpixels are picked among the particular pixels based on the differencebetween the adjacent blocks (step 150). Finally, the peripheral areasaround the scratch pixels are subdivided and filled up (step 160).

FIG. 2 is a flowchart of a method of repairing scratches on a digitalimage according to one embodiment of the invention. The images arepre-processed (step 210). Practically, the crude image acquired is notperfect due to noise, reflection or illumination. Therefore,pre-processing is necessary. Pre-processing of the crude images includesimage brightening, filtering and sharpening. Image filtering eliminatesor weakens the spots, stains, scratches, and voids, etc. to improve theimage quality.

The images acquired by sensors are constituted by RGB values widely usedin color systems. The RGB are major colors with respect to people'svisual sense for color representation. In the invention, the RBG modeldata are converted into a YcbCr format data (step 220), wherein Y isbrightness, and Cb and Cr respectively represent hue and saturation.Here, only Y value is obtained.

Converting RGB format to YcbCr format are common data conversions thatuse a conversion coefficient. The conversion is accomplished as follows:Y=0.2990R+0.5870G+0.1140BCb=−0.1687R−0.3313G+0.5000B+128Cr=0.5000R−0.4187G−0.0813B+128

Precise localization of scratches is crucial to scratch processing.Scratch localization distinguishes scratch areas on the image and marksthem as scratch pixels. Each pixel corresponds to one Y valuerepresenting its brightness. The brightness of scratch pixels is greaterthan that of normal pixels. Therefore, the brighter pixels are picked as‘selected pixels’. The brighter pixels have Y values greater than apredetermined brightness value, for example 80 in this embodiment. Thatis, the pixels having an Y value greater than 80 are the selected pixels(step 230).

Referring to FIG. 3, each selected pixel is surrounded by 8 neighboringpixels spaced uniformly. The following table illustrates a groupincluding the target pixel and the neighboring pixels:

Y (i − 1, j − 1) Y (i − 1, j) Y (i − 1, j + 1) Y (i, j − 1) Y (i, j) Y(i, j + 1) Y (i + 1, j − 1) Y (i + 1, j) Y (i + 1, j + 1)

Here below is an example according to the above table.Y(i−1,j−1)Y(i−1,j)Y(i−1,j+1)Y=Y(i,j−1)Y(i,j)Y(i,j+1)=Y(i+1,j−1)Y(i+1,j)Y(i+1,j+1)

217.0273 200.8299 122.9947 206.0808 161.8376 108.1474 180.5265 109.314588.5259

There may be abnormal pixels among these neighboring pixels. For onegiven selected pixel, the brightness difference between each abnormalpixel and the selected pixel is greater than that between each of thenormal neighboring pixels and the selected pixel. All the abnormalpixels have to be found and marked. The way to find the abnormal pixelsis to calculate an absolute value of brightness difference between eachneighboring pixel and their corresponding selected pixel. If theabsolute value is greater than an abnormal standard, then thecorresponding neighboring pixel is marked as an abnormal pixel. In thisembodiment, the abnormal standard is 40, and any neighboring pixelhaving an absolute value greater than 40 is marked as ‘abnormal pixel’(step 240).

Thereafter, abnormal pixels around each selected pixel are summed up. Ifthe resulting number of abnormal pixels around each selected pixel isgreater than a predetermined value, the selected pixel is designated asa particular pixel. The predetermined value is, for example, 4. All theparticular pixels are found according to the above manner (step 250).

The particular pixels are the selected pixels having greater brightnessand greater amount of abnormal neighboring pixels. The particular pixelsmay be either scratch pixels or edge pixels. Now, the scratch pixels aredistinguished from the edge pixels. The edge pixels are the pixels withthe greatest change in brightness at certain locations of the image. Incontrast to the edge pixels, the contrast between one scratch pixel andits neighboring pixels is small. The area surrounding each particularpixel is divided into a plurality of blocks, for example, 3, 4 or 8blocks. In the illustration of FIG. 4A to FIG. 4D, the surrounding areais divided into 4 blocks respectively designated as block 401, block402, block 403 and block 404, each block including 9 neighboring pixels.A mean square error of 9 neighboring pixels of each block for eachparticular pixel is estimated. Then calculate the mean square errordifference of the 4 blocks (step 260). The mean square error differenceis calculated according to the following expression.Dif=MaxVar−Mid1Var−Mid2Var+MinVar

whereinDif is a difference of the mean sqaure error, MaxVar is a maximummean square error, MinVar is a minimum mean square error, and Mid1Varand Mid2Var are middle values among the four mean square error values.If a particular pixel has a Dif smaller than a difference standard, thenit is a scratch pixel. On the other hand, if a particular pixel has aDif value greater than the difference standard, then it is a edge pixel.The difference standard is in the range of 5˜8. In this embodiment, thedifference standard is 5.5. That is if MaxVar−Mid1Var−Mid2Var+MinVar<5.5(step 270), the particular pixel is designated as a scratch pixel (step280), otherwise as a edge pixel (step 281).

When the number of blocks is 3, then the expression used to calculateDif value is: MaxVar−2Mid1Var+MinVar. In contrast to the situation of 4blocks, MidVar is a middle among the three mean square error values.

After the scratch pixel is determined, the scratch can be foundaccording to the areas surrounded by the scratch pixels. The areasurrounded by the scratched pixels usually need to be furthersubdivided. Morph technology is a pixel interpolation method widely usedin two-dimensional animation, especially in movie trick shots. Morphtechnology makes up different film sections as a continuous frame. Theinvention uses Morph technology to subdivide the area surrounded by thescratch pixels and then Convert YcbCr format to RGB format. At last inRGB format fill up the subdivided areas (step 290). In filling up thesubdivided areas, mean values of the pixels surrounding the scratchreplace that of the scratch to achieve image repairing.

Knowing the invention being thus described, it will be obvious that thesame may be varied in many ways. Such variations are not to be regardedas a departure from the spirit and scope of the invention, and all suchmodifications as would be obvious to one skilled in the art are intendedto be included within the scope of the following claims.

1. A method of repairing scratches in digital images, the method comprising: calculating a brightness difference between a selected pixel and one of neighboring pixels surrounding the selected pixel; counting the number of abnormal pixels among the neighboring pixels; designating as particular pixels the selected pixels surrounded by the abnormal pixels, wherein the number of abnormal pixels is more than a predetermined value; dividing an area surrounding each particular pixel into a plurality of blocks, and calculating a brightness difference between the blocks; finding out scratch pixels from the particular pixels based on brightness difference between blocks; and subdividing and filling up an area surrounded by scratch pixels, wherein the step of calculating brightness difference between the blocks includes estimating a mean square error of each block which has a plurality of pixels, and calculating the difference between the mean square errors.
 2. The method of claim 1, wherein a neighboring pixel is an abnormal pixel when an absolute value of the brightness difference between the neighboring pixel and its corresponding selected pixel is greater than an abnormal standard.
 3. The method of claim 1, wherein the particular pixels include scratch pixels and edge pixels.
 4. The method of claim 1, wherein the area surrounding each particular pixel is divided into 4 blocks.
 5. The method of claim 1, wherein the area surrounding each particular pixel is divided into 3 blocks.
 6. The method of claim 1, wherein the area surrounding each particular pixel is divided into 8 blocks.
 7. The method of claim 1, wherein if the number of blocks is 4, then the difference between the mean square errors is obtained according to the following expression: Dif=MaxVar−2MidVar+MinVar, wherein Dif is a difference of the mean square errors, MaxVar is a maximum mean square error, MinVar is a minimum mean square error, and MidVar is a middle among the three mean square error values.
 8. The method of claim 1, wherein if the number of blocks is 3, then the difference between the mean square errors is obtained according to the following expression: Dif=MaxVar−2MidVar+MinVar, wherein Dif is a difference of the mean sqaure errors, MaxVar is a maximum mean square error, MinVar is a minimum mean square error, and MidVar is a middle among the three mean square error values.
 9. The method of claim 1, wherein at the step of finding out scratch pixels from the particular pixels based on brightness difference between blocks, if the particular pixel has a Dif smaller than a difference standard, then the particular pixel is a scratch pixel, otherwise the particular pixel is an edge pixel. 